Genetic Programming Framework Research Articles

A Novel Multiobjective Genetic Programming Approach to High-Dimensional Data Classification.

The development of data sensing technology has generated a vast amount of high-dimensional data, posing great challenges for machine learning models. Over the past decades, despite demonstrating its effectiveness in data classification, genetic programming (GP) has still encountered three major challenges when dealing with high-dimensional data: 1) solution diversity; 2) multiclass imbalance; and 3) large feature space. In this article, we have developed a problem-specific multiobjective GP framework (PS-MOGP) for handling classification tasks with high-dimensional data. To reduce the large solution space caused by high dimensionality, we incorporate the recursive feature elimination strategy based on mining the archive of evolved GP solutions. A progressive domination Pareto archive evolution strategy (PD-PAES), which optimizes the objectives in a specific order according to their objectives, is proposed to evaluate the GP individuals and maintain a better diversity of solutions. Besides, to address the seriously imbalanced class issue caused by traditional binary decomposition (BD) one versus rest (OVR) for multiclass classification problems, we design a method named BD with a similar positive and negative class size (BD-SPNCS) to generate a set of auxiliary classifiers. Experimental results on benchmark and real-world datasets demonstrate that our proposed PS-MOGP outperforms state-of-the-art traditional and evolutionary classification methods in the context of high-dimensional data classification.

A surrogate-assisted dual-tree genetic programming framework for dynamic resource constrained multi-project scheduling problem

Genetic programming has achieved great success in project scheduling by generating Priority Rules (PRs) through evolution. However, the frequent disturbance factors in practice not only lead to the appropriate PR changes in different states, but also increase the calculation consumption in evaluation. In this paper, a novel Hyper-heuristic-based Surrogate-Assisted Dual-Tree Genetic Programming (HSDGP) framework is proposed for the Dynamic Resource Constrained Multi-Project Scheduling Problem with new project Insertions and resource Disruptions (DRCMPSP-ID). Uniquely, the proposed method automatically evolves two PRs for scheduling DRCMPSP-ID under normal and disruptive states respectively, and an expansion search mechanism based on neighbourhood is designed to improve PR generation ability by generating a large number of offspring and implement the search of dual-tree encoding. Furthermore, in order to estimate the fitness of new individuals, an activity-sequence based surrogate is proposed to deal with the input of activity sequence during schedule generation and reduce the evaluation calculation consumption. Based on the instances constructed by the existing benchmark, the experimental result shows the superiority of HSDGP and the impact of key parameters on its performance.

Automatically evolving preference-based dispatching rules for multi-objective job shop scheduling

Dispatching rules represent a simple heuristic for finding good solutions for job shop scheduling problems. Due to their fast applicability and easy handling, they are often used in manufacturing companies to create appropriate production schedules. It has been shown that dispatching rules that are specifically designed for the requirements of a particular environment improve the performance of schedules. Hyper-heuristics based on genetic programming can be used for the automated generation of such dispatching rules. Evolutionary algorithms search the space of dispatching rule components for the most effective priority function to optimize the performance of the resulting schedule. Various studies have highlighted the advantages in the single-objective case, which made it possible to derive a large number of new dispatching rules that exceeded previous benchmark rules. Because it is usually necessary to consider more than one objective simultaneously to ensure effective creation of schedules, the need for a multi-objective optimization method arises. In this paper, we propose an interactive multi-objective optimization method, namely the reference point method, implemented in a hyper-heuristic genetic programming framework. A decision support system has also been developed and implemented in a web-based application to facilitate interaction with the user. Incorporating preferences into the solution process aims to efficiently evolve a dispatching rule that meets the expectations of a decision-maker. A fictitious experiment was carried out in a benchmark job shop environment. The results show that the final solution selected by the decision-maker can produce schedules achieving a desired compromise between the makespan, total tardiness, and total waiting time. Testing the evolved dispatching rule on an independent set of instances and comparing its performance with other benchmark dispatching rules revealed that the proposed method successfully finds dispatching rules that meet the decision-maker’s expectations and are capable of reproducing similar compromise schedules for unseen problems in the same environment.

A new fitness function in genetic programming for classification of imbalanced data

ABSTRACT Many real-world problems have an uneven distribution of data over different classes. The imbalanced nature of data impacts the performance of classifiers. The higher counts of majority class samples influence the learning abilities of well-known classifiers. Genetic programming (GP) algorithm based on natural evolution also impacts if the data’s nature is imbalanced. The fitness function plays a pivotal role and impacts almost each building block of the GP framework. GP with the standard fitness function produces under-fitted and biased classifiers. Therefore, this paper has proposed a new fitness function in GP to classify the imbalanced data. The proposed method is used to classify nine imbalanced problems: ABL-18, ABL-9-18, BAL, YEAST2, YEAST1, ABL-9, ION, WDBC, and SPECT. The imbalanced factor of benchmark problems varies from 99:1 to 59:41. The proposed method’s performance is compared with K-Nearest-Neighbourhood (KNN) and the standard fitness function-based GP methods. The GP with newly proposed fitness function gives average AUC values for ABL-18(99:1), ABL-9-18(94:6), BAL(92:8), YEAST2(89:11), YEAST1(84:16), ABL-9(83:17), ION(64:36), WDBC(63:37), and SPECT(59:41) as 0.714, 0.812, 0.975, 0.916, 0.768, 0.654, 0.872, 0.939, and 0.704, respectively, which are higher than KNN and the standard fitness function-based GP methods. The result outcomes prove the superiority of the proposed method.

A two-stage genetic programming framework for Stochastic Resource Constrained Multi-Project Scheduling Problem under New Project Insertions

This study proposes a novel hyper-heuristic based two-stage genetic programming framework (HH-TGP) to solve the Stochastic Resource Constrained Multi-Project Scheduling Problem under New Project Insertions (SRCMPSP-NPI). It divides the evolution of genetic programming into generation and selection stages, and then establishes a multi-state combination scheduling mode with multiple priority rules (PRs) for the first time to realize resource constrained project scheduling under both stochastic activity duration and new project insertion. In the generation stage, based on a modified attribute set for multi-project scheduling, NSGA-II is hybridized to evolve a non-dominated PR set for forming a selectable PR set. While in the selection stage, the whole decision-making process is divided into multiple states based on the completion activity duration, and a weighted normalized evolution process with two crossovers, two mutations and four local search operators to match the optimal PR for each state from the PR set. Under the existing benchmark, HH-TGP is compared with the existing methods to verify its effectiveness.

A filtering genetic programming framework for stochastic resource constrained multi-project scheduling problem under new project insertions

Multi-project management and uncertain environment are very common factors, and they bring greater challenges to scheduling due to the increase of problem complexity and response efficiency requirements. In this paper, a novel hyper-heuristic based filtering genetic programming (HH-FGP) framework is proposed for evolving priority rules (PRs) to deal with a multi-project scheduling problem considering stochastic activity duration and new project insertion together, namely the Stochastic Resource Constrained Multi-Project Scheduling Problem under New Project Insertions (SRCMPSP-NPI), within heuristic computation time. HH-FGP is designed to divide traditional evolution into sampling and filtering evolution for simultaneously filtering two kinds of parameters constituting PRs, namely depth range and attribute, to obtain more effective PRs. Based on this, the existing genetic search and local search are improved to meet the depth constraints, and a multi-objective evaluation mechanism is designed to achieve effective filtering. Under the existing benchmark, HH-FGP is compared and analysed with the existing methods to verify its effectiveness.

A semantic genetic programming framework based on dynamic targets

A semantic genetic programming framework based on dynamic targets

A genetic programming-based feature selection and fusion for facial expression recognition

Emotion recognition has become one of the most active research areas in pattern recognition due to the emergence of human–machine interaction systems. Describing facial expression is a very challenging problem since it relies on the quality of the face representation. A multitude of features have been proposed in the literature to describe facial expression. None of these features is universal for accurately capturing all the emotions since facial expressions vary according to the person, gender and type of emotion (posed or spontaneous). Therefore, some research works have considered combining several features to enhance the recognition rate. But they faced significant problems because of information redundancy and high dimensionality of the resulting features. In this work, we propose a genetic programming framework for feature selection and fusion for facial expression recognition, which we called GP−FER. The main component of this framework is a tree-based genetic program with a three functional layers (feature selection, feature fusion and classification). The proposed genetic program is a binary classifier that performs discriminative feature selection and fusion differently for each pair of expression classes. The final emotion is captured by performing a unique tournament elimination between all the classes using the binary programs. Three different geometric and texture features were fused using the proposed GP−FER. The obtained results, on four posed and spontaneous facial expression datasets (DISFA, DISFA+, CK+ and MUG), show that the proposed facial expression recognition method has outperformed, or achieved a comparable performance to the state-of-the-art methods.

Generating and Optimizing Human-Readable Quantitative Program Trading Strategies through a Genetic Programming Framework

In this paper, we provide a highly flexible genetic programming framework for automatic generation and optimization of program trading strategies. We propose the input/output modules and their implementation methods, decoupled from the GP kernel, making it a priori-posteriori framework for trading practitioners. For human-readable purposes, we also give various empirical regularization methods, including NSGA-II multi-objective selection, as well as experimentally effective performance measures.

An efficient memetic genetic programming framework for symbolic regression

Symbolic regression is one of the most common applications of genetic programming (GP), which is a popular evolutionary algorithm in automatic computer program generation. Despite existing success of GP on symbolic regression, the accuracy and efficiency of GP can still be improved especially on complicated symbolic regression problems, enabling GP to be applied to more fields. This paper proposes a novel memetic GP framework to improve the accuracy and search efficiency of GP on complicated symbolic regression problems. The proposed framework consists of two components: feature construction and feature combination. The first component focuses on constructing diverse features. The second component aims to filter redundant features and linearly combines these independent features. The first component (feature construction) focuses on constructing polynomial features derived from polynomial functions, and evolves features by a GP solver. In addition, a gradient-based nonlinear least squares algorithm named Levenberg-Marquardt (LM) is embedded in the second component (feature combination) to locally adjust the weights of independent features. A filtering mechanism is put forward to discard redundant features in the second component. Hence, the polynomial features and evolved features can work together in the framework to improve the performance of GP. Experimental results demonstrate that the proposed framework offers enhanced performance compared with several state-of-the-art algorithms in terms of accuracy and search efficiency on nine benchmark regression problems and three real-world regression problems. In this study, a novel memetic genetic programming framework is proposed to improve the performance of GP on symbolic regression. Experimental results demonstrate that the proposed framework can improve the accuracy and search efficiency of GP on complicated symbolic regression problems compared with four state-of-the-art algorithms.

Framework of model selection criteria approximated genetic programming for optimization function for renewable energy systems

Abstract For the realization of complex renewable energy systems (such as nano-fluids based direct absorption solar collector), an evolutionary system identification method such as genetic programming (GP) can be applied to develop mathematical models/functional relationships between the process parameters. The system complexity is attributed to interaction among the design variables influencing the outputs. There are also uncertainties in the system due to random and unknown variations in the design and response variables. GP suffers from the higher complexity structure of its solutions and non-optimal convergence, which leads to poor fitness values. Therefore, to address these uncertainties and problems, the framework based on the model selection criteria approximated genetic programming (MSC-GP) is proposed for the formulation of geometry design based thermal efficiency and entropy generation optimization function for direct absorption solar collector (DASC) system. In this proposed method, the four mathematical model selection criteria are used as an approximation for objective functions in GP framework for the evaluation of fitting degree and structure of the model. The results based on statistical measures (best fitness, mean fitness, standard deviation of fitness, number of nodes) show that models obtained from the mathematical selection criteria, Predicted Residual error sum of squares (PRESS), have performed the best. Based on Pareto front analysis of PRESS function, it is found that the best objective values and the number of nodes of models (complexity) follows more or less gradually slow increasing trend which is a good symbolic desirable sign of minimal increase of complexity of model with a decrease in objective values as the values of generation increases. The results of the sensitivity analysis show that the main factor affecting the efficiency of DASC is its geometry of the structure. 3-D interaction analysis shows that increasing the thickness, length and reducing the width of the collector can make the system maintain its higher thermal efficiency and a smaller entropy generation, which is useful for the optimized operation of DASC. Non-dominated sorting genetic algorithm-II (NSGA-II) is applied in the acquisition of the optimal geometric settings of DASC system based on the selected models. The optimal settings achieved is 5 cm in length, 5 cm in width, and 2 cm in thickness. Systems when operated using these settings results in a satisfactory performance with 77.8117% in thermal efficiency and 6.0004E+3 in entropy generation).

A Soft Computing Approach for Selecting and Combining Spectral Bands

We introduce a soft computing approach for automatically selecting and combining indices from remote sensing multispectral images that can be used for classification tasks. The proposed approach is based on a Genetic-Programming (GP) framework, a technique successfully used in a wide variety of optimization problems. Through GP, it is possible to learn indices that maximize the separability of samples from two different classes. Once the indices specialized for all the pairs of classes are obtained, they are used in pixelwise classification tasks. We used the GP-based solution to evaluate complex classification problems, such as those that are related to the discrimination of vegetation types within and between tropical biomes. Using time series defined in terms of the learned spectral indices, we show that the GP framework leads to superior results than other indices that are used to discriminate and classify tropical biomes.

Learning feature spaces for regression with genetic programming.

Genetic programming has found recent success as a tool for learning sets of features for regression and classification. Multidimensional genetic programming is a useful variant of genetic programming for this task because it represents candidate solutions as sets of programs. These sets of programs expose additional information that can be exploited for building block identification. In this work, we discuss this architecture and others in terms of their propensity for allowing heuristic search to utilize information during the evolutionary process. We investigate methods for biasing the components of programs that are promoted in order to guide search towards useful and complementary feature spaces. We study two main approaches: 1) the introduction of new objectives and 2) the use of specialized semantic variation operators. We find that a semantic crossover operator based on stagewise regression leads to significant improvements on a set of regression problems. The inclusion of semantic crossover produces state-of-the-art results in a large benchmark study of open-source regression problems in comparison to several state-of-the-art machine learning approaches and other genetic programming frameworks. Finally, we look at the collinearity and complexity of the data representations produced by different methods, in order to assess whether relevant, concise, and independent factors of variation can be produced in application.

A novel fitness function in genetic programming to handle unbalanced emotion recognition data

In the area of behavioral psychology, real-time emotion recognition by using physiological stimuli is an active topic of interest. This research considers the recognition of two class of emotions i.e., positive and negative emotions using EEG signals in response to happy, horror, sad, and neutral genres. In a noise-free framework for data acquisition of 50 participants, NeuroSky MindWave 2 is used. The dataset collected is unbalanced i.e., there are more instances of positive classes than negative ones. Therefore, accuracy is not a useful metric to assess the results of the unbalanced dataset because of biased results. So, the primary goal of this research is to address the issue of unbalanced emotion recognition dataset classification, for which we are proposing a novel fitness function known as Gap score (G score), which learns about both the classes by giving them equal importance and being unbiased. The genetic programming (GP) framework in which we implemented G score is named as G-score GP (GGP). The second goal is to assess how distinct genres affect human emotion recognition process and to identify an age group that is more active emotionally when their emotions are elicited. Experiments were conducted on EEG data acquired with a single-channel EEG device. We have compared the performance of GGP for the classification of emotions with state-of-the-art methods. The analysis shows that GGP provides 87.61% classification accuracy by using EEG. In compliance with the self-reported feelings, brain signals of 26 to 35 years of age group provided the highest emotion recognition rate.

Multi-objective optimisation framework of genetic programming for investigation of bullwhip effect and net stock amplification for three-stage supply chain systems

In this work a multi-objective optimisation framework of genetic programming (GP) in the modelling of bullwhip effect and NSA for centralised and decentralised supply chain systems has been proposed. The individual and interactive effect of these four input factors has been investigated on bullwhip effect and NSA by adapting the parametric and sensitivity approach on the formulated models. The appropriate settings of dominant input factors (batch ordering and demand signal processing for a decentralised chain, demand signal processing and rationing shortage gaming for a centralised chain) are suggested to optimise the bullwhip effect and NSA of three-stage supply chain simultaneously. The implications and advantages of proposed optimisation framework will be useful for business practitioners to monitor and supervise the sudden demand amplification that generally faced by them in the supply chains.

Multi-objective optimisation framework of genetic programming for investigation of bullwhip effect and net stock amplification for three-stage supply chain systems

Multi-objective optimisation framework of genetic programming for investigation of bullwhip effect and net stock amplification for three-stage supply chain systems

A fast parallel genetic programming framework with adaptively weighted primitives for symbolic regression

Genetic programming (GP) is a popular and powerful optimization algorithm that has a wide range of applications, such as time series prediction, classification, data mining, and knowledge discovery. Despite the great success it enjoyed, selecting the proper primitives from high-dimension primitive set for GP to construct solutions is still a time-consuming and challenging issue that limits the efficacy of GP in real-world applications. In this paper, we propose a multi-population GP framework with adaptively weighted primitives to address the above issues. In the proposed framework, the entire population consists of several sub-populations and each has a different vector of primitive weights to determine the probability of using the corresponding primitives in a sub-population. By adaptively adjusting the weights of the primitives and periodically sharing information between sub-populations, the proposed framework can efficiently identify important primitives to assist the search. Furthermore, based on the proposed framework and the graphics processing unit computing technique, a high-performance self-learning gene expression programming algorithm (HSL-GEP) is developed. The HSL-GEP is tested on fifteen problems, including four real-world problems. The experimental results have demonstrated that the proposed HSL-GEP outperforms several state-of-the-art GPs, in terms of both solution quality and search efficiency.

Unbalanced breast cancer data classification using novel fitness functions in genetic programming

Breast Cancer is a common disease and to prevent it, the disease must be identified at earlier stages. Available breast cancer datasets are unbalanced in nature, i.e. there are more instances of benign (non-cancerous) cases then malignant (cancerous) ones. Therefore, it is a challenging task for most machine learning (ML) models to classify between benign and malignant cases properly, even though they have high accuracy. Accuracy is not a good metric to assess the results of ML models on breast cancer dataset because of biased results. To address this issue, we use Genetic Programming (GP) and propose two fitness functions. First one is F2 score which focuses on learning more about the minority class, which contains more relevant information, the second one is a novel fitness function known as Distance score (D score) which learns about both the classes by giving them equal importance and being unbiased. The GP framework in which we implemented D score is named as D-score GP (DGP) and the framework implemented with F2 score is named as F2GP. The proposed F2GP achieved a maximum accuracy of 99.63%, 99.51% and 100% for 60-40, 70-30 partition schemes and 10 fold cross validation scheme respectively and DGP achieves a maximum accuracy of 99.63%, 98.5% and 100% in 60-40, 70-30 partition schemes and 10 fold cross validation scheme respectively. The proposed models also achieves a recall of 100% for all the test cases. This shows that using a new fitness function for unbalanced data classification improves the performance of a classifier.

An integrated framework of genetic network programming and multi-layer perceptron neural network for prediction of daily stock return: An application in Tehran stock exchange market

Evolutionary algorithms are generally used to find or generate the best individuals in a population. Whenever these algorithms are applied to agent systems, they will lead to optimal solutions. Genetic Network Programming (GNP), which contains graph networks, is one of the developed evolutionary algorithms. When the aim is to forecast the share price or return, ascending and descending trends, volatilities, recent returns, fundamental and technical factors have remarkable impacts on the prediction. This is why technical indicators are used to constitute a set of trading rules. In this paper, we apply an integrated framework consisting of GNP model along with a reinforcement learning and Multi-Layer Perceptron (MLP) neural network to classify data and also time series models to forecast the stock return. Moreover, we utilize rules of accumulation based on the GNP model’s results to forecast the return. The aim of using these models alongside one another is to estimate one-day return. The results derived from 9 stocks with regard to the Tehran Stock Exchange Market. GNP extracts a prodigious number of rules on the basis of 5 technical indicators with 3 times period. Next, MLP network classifies data and finds the similarity between future data and past data concerning a stock (5 sub-period) through classification. Subsequently, a number of conditions are established, in order to choose the best estimation between GNP-RL and ARMA. Distinct comparison with the ARMA–GARCH model, which is operated for return estimation and risk measurement in many researches, demonstrates an extended forecasting power of the proposed model, by the name of GNP–ARMA, reducing error by a mean of 16%.

Evolutionary framework design in formulation of decision support models for production emissions and net profit of firm: Implications on environmental concerns of supply chains

There have been increased investments in cleaner technologies and adoption of a voluntarily limit on transportation emissions by the global firms to handle the environmental concerns of supply chains and to increase demand for finished goods. Consequences are the reduction in net profit for the firm. To address this trade-off between the net profit and environmental concerns, the formulation and optimization of a compact model are needed. Development of these models requires a thorough understanding of the nature of the impact of three inputs (investment coefficient, penalty per unit emission and customer's emission elasticity) on production emissions and net profit. Past studies revealed that a compact model comprising the interactive effect of these inputs on the production emissions and net profit is not yet formulated. Therefore, this study illustrates the development of an evolutionary framework of an advanced multi-gene genetic programming in the formulation of functional expressions for the net profit and production emissions based on the three inputs (investment coefficient, penalty per unit emission and customer's emission elasticity) of the monopolist firm. The sensitivity and parametric based 2-D analysis determine the relationships and found that the penalty per unit emission is dominant input for reducing emissions and maintaining net profit simultaneously. The contribution of this work lies in designing of an evolutionary framework in the development of empirical explicit expressions, which can easily be optimized analytically to keep production emissions and net profit balanced.